Towards an Embedded Visuo-Inertial Smart Sensor

In neurological system of primates, changes in posture are detected by the central nervous system through a vestibular process. This process, located in inner ear, coordinates several system outputs to maintain stable balance, visual gaze, and autonomic control in response to changes in posture. Consequently the vestibular data is merged to other sensing data like touch, vision,.... The visuo-inertial merging is crucial for several tasks like navigation, depth estimation, stabilization. This paper proposes a ”primate-inspired” sensing hardware, based on a CMOS imaging and an artificial vestibular system. The whole sensor can be considered like a smart embedded sensor where one of the most original aspect of this approach is the use of a System On Chip implemented in a FPGA to manage the whole system. The sensing device is designed around a 4 Millions pixels CMOS imager and the artificial vestibular set is composed by 3 linear accelerometers and 3 gyroscopes. With its structure, the system proposes a high degree of versatility and allows the implementation of parallel image and inertial processing algorithms. In order to illustrate the proposed approach, we consider a depth estimation with Kalman filtering implementation.